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Key findings 


e Using math and reading test scores from 2.1 million Black, Indigenous, and people of color 
(BIPOC) students in grades 3 to 8, we found that math achievement was significantly impacted 
in fall 2020 but reading achievement was mostly parallel to prior years. 

e BIPOC students’ math achievement during the pandemic dropped relative to a pre-pandemic 
national sample of same-grade students, particularly for students who were in 4", 5", and 6" 
grade in fall 2020. 

e All student groups exhibited average test score gains during the fall 2019 to fall 2020 period, 
indicating that students were learning during the pandemic, however the rate of learning was 
lower than during the pre-pandemic period. 

e Black, American Indian and Alaska Native, and Latinx students made 55-68% of typical learning 
gains in math and 73-86% of typical gains in reading during the pandemic, whereas Asian 
American students made between 88-100% of their typical gains in the two subjects. 

e Male BIPOC students and BIPOC students attending high-poverty schools were the least likely 
to achieve typical learning gains during the pandemic. 


Introduction 


The COVID-19 pandemic has severely disrupted educational systems across the U.S., prompting 
concerns about a “lost year” of schooling for many children. |nitial evidence from NWEA assessments 
indicates that students’ math test scores decreased substantively relative to students testing in the prior 
school year. Further, a smaller proportion of students made gains in math during COVID-19 than in the 
year prior for all grades. For example, 57% of students starting in 3 grade made academic gains 
between winter 2020 and fall 2020 (during a period of school closures), relative to 79% of students 


between winter 2019 and fall 2019. While reading 
achievement and growth were fairly similar before 
and after the start of the COVID-19 pandemic in 
our own study of 3% through 8" graders, other 
studies have found that students in earlier grades 
struggled more in reading since the onset of the 
pandemic. There is also a possibility that results to 
date understate the true negative impact of COVID- 
19 on achievement. A key finding from our 
research was that approximately one in four 
students who had tested last year were missing in 
the fall 2020 data, and that a higher percentage of 
missing test takers were Black or Latinx and 
attended schools serving a higher proportion of 
students living in poverty. Thus, our earlier study 
likely did not fully capture the outcomes of the very 
students that were hit hardest by the pandemic and 
the ongoing disruptions to learning it has wrought. 


Framing 


We acknowledge that enumerating the 
educational inequities faced by BIPOC 
students and the ways BIPOC communities 
have been more adversely affected by the 
pandemic may be seen as adopting a deficit- 
based perspective. This perspective often 
perpetuates a victim-blaming orientation. 
However, we do not believe that BIPOC 
students need “fixing;” focus should instead 
be on the dominant power systems that 
continue to provide inequitable supports for 
BIPOC students. 


In this report, we address this incomplete understanding by looking more closely at data on the impact’ 
of the pandemic on Black, Indigenous, and people of color (BIPOC)? students. BIPOC students have 
faced longstanding systematic educational inequalities that have been exacerbated during the 
pandemic. Recent data collected by the U.S. Department of Education indicate that BIPOC students 
were twice as likely to be learning remotely than White students in January 2021. Remote learning puts 
more of the teaching and learning burden on students and their guardians, making it even harder for 
them to access the academic content they need during the pandemic. Further complicating matters, 
Black and Latinx children were less likely than White children to have a parent who could work from 
home during the pandemic, meaning they had to manage their own learning to a greater extent than 
their peers. Emergent bilingual students also likely experienced additional barriers to learning as they 
tried to manage their learning in a language they are still learning. 


While such barriers to learning for BIPOC students during the pandemic are immense, they at least 
assume that students can access on-line content. However, due to decades long patterns of 
discrimination in housing policy and differential access to technology infrastructure, many BIPOC 
students live in homes without computers/devices, internet, and/or dedicated learning spaces—all 
essential for effective online teaching and learning. For example, over one-third of American Indian and 
Alaska Native (AIAN) children live in households without broadband internet access (the highest of all 
groups), and 15% live in homes without a computer. In short, students’ ability to even access online 
content was itself inequitable. 


The unequal impacts of the pandemic on BIPOC communities extend far beyond education. For 
instance, the pandemic highlights systematic inequalities in access to high-quality medical care, 
participation in the workforce, and mortality among individuals in BIPOC communities. These 
inequalities manifest in higher rates of unemployment among Black households and higher COVID-19 
infection and death rates in the Black community. In addition, Latinx and AIAN individuals have three 
times the risk of hospitalization and two times the risk of death from COVID-19 as White individuals. In 
the workforce, Latinx workers are among those most affected by the pandemic with nearly 50% being 
forced to take a pay cut and/or laid off. It seems likely that these differential health outcomes and 
economic impacts will only serve to further impact academic achievement. 


Evidence is also emerging about the effect of the pandemic on mental health, particularly for BIPOC 
youth. A recent CRPE report based on the YouthTruth survey found that nearly half of all students who 
responded reported elevated levels of depression, stress, and anxiety. Further, an analysis of youth 
visiting a pediatric emergency department found a stark increase in the prevalence of suicide risk 
among youth aged 11-21 during the pandemic compared to the year prior. Mental health issues may be 
taking an even greater toll on BIPOC students. For example, according to a Common Sense media 
poll, Black children are particularly vulnerable to mental health challenges during the COVID-19 
pandemic, with 74% of Black respondents concerned about the effect that COVID-19 may have on their 
family’s finances and 71% worried that they or a family member will contract the virus. Stressors related 
to COVID-19 may be compounded by societal unrest in the face of race-related traumas, including 
those involving police brutality towards Black people and a rise in overt acts of discrimination against 
Asian Americans. Sixty percent of Asian Americans witnessed someone blame Asian people for the 
pandemic, and over 80% of Chinese American teenagers reported that they experienced or witnessed 
COVID-19-related discrimination in person or online. Trauma and stress from police shootings, 


1 This study provides descriptive evidence on contrasts in pre- and post-pandemic achievement trends. We use words such as 
impact and effect for simplicity, not to suggest causal warrant. 

2 BIPOC are anyone who identifies as Black, American Indian and Alaska Native (AIAN), Native Hawaiian, Asian American, 
Latinx, and Pacific Islander. 


violence, and discrimination could exacerbate the educational impact of COVID-19, especially students 
and families of color. 


BIPOC students and communities are severely affected by the COVID-19 virus and bear the brunt of its 
psychological and socioeconomic effects, compounding the impact of the educational inequities BIPOC 
students face. Student achievement and growth for BIPOC students during the pandemic will reflect the 
cumulative effect of these inequities. The purpose of this report is to document achievement trends for 
BIPOC students and provide policymakers with evidence to guide action to address these inequities. 
Specifically, we examine the academic trajectories of Black, Latinx, Native Hawaiian, Asian American, 
and AIAN students during the pandemic relative to a typical school year and seek to answer the 
following research questions: 


1) How does the academic achievement of BIPOC students during the pandemic compare to 
national pre-pandemic patterns of achievement? 

2) How do learning gains of BIPOC students from fall 2019 to fall 2020 compare to BIPOC student 
gains in prior years? 


We analyzed data® from 2.1 million* BIPOC 


About MAP Growth 


The NWEA MAP Growth assessment provides 
precise estimates of student achievement status 
(what students know and can do) as well as growth 
over time (the amount of learning gains made by 
students since they were last assessed). In addition, 
NWEA nationally representative norms (which were 
estimated before the COVID-19 pandemic) 
contextualize how a student’s achievement and 
growth compares to academic peers. These 
features of the MAP Growth assessment make it 
possible to address research questions about 
student achievement status (RQ1) and growth over 


students in grades 3 to 8 who took the MAP 
Growth assessments in reading and math. 
We use a pre-pandemic sample of students 
as a benchmark to measure how the 
pandemic has changed students’ learning. 
The benchmark statistic that we used was a 
student’s percentile rank, which situates 
student performance relative to a pre- 
pandemic national norming sample. To 
address the first research question, we 
compared the achievement of BIPOC 
students in the fall of 2020 to their 
achievement in the fall of 2019 (see “About 


time (RQ2). MAP Growth’ text box, left). This analysis 
illustrates how BIPOC students’ percentile 
rank in the fall of 2020 compares to their 
own prior pre-pandemic percentile rank (fall 
2019) as well as to overall national pre-pandemic averages (calculated based on test scores from all 


student subgroups). 


BIPOC students’ math achievement was substantially lower in fall 2020 than it was in fall 2019, 
but reading performance showed only small declines during the same period. 


Figure 1 shows fall-to-fall percentile rank changes in math by grade level for students in the five BIPOC 
groups. Students are followed across two grade levels between fall 2019 and fall 2020 (for instance, 3° 


3 Overall, the sample is 4% AIAN, 10% Asian American, 41% Black, 44% Latinx, and 1% Native Hawaiian. In line with Critical 
Race and Critical Whiteness Theorists, we explicitly chose to exclude White students from our analyses to avoid centering 
whiteness as the default academic standard. For more details on this decision, see the appendix section. 

“To account for differential patterns of missing data in our study, we multiply imputed scores for students who tested in 2019- 
20 but were not observed in fall 2020. See the appendix for more detail on the multiple imputation strategy. 


graders in the fall of 2019 were followed to the fall of 2020 as they entered 4" grade). Math 
performance dropped across nearly all grades and groups relative to pre-pandemic levels. For 
example, 4'" grade Black and AIAN students were each at approximately the 40" percentile before the 
COVID-19 pandemic and dropped about 10 percentile points by fall 2020. This means that in fall 2019, 
4" grade Black and AIAN students’ scores were greater than 40% of a national pre-pandemic sample 
of students, but by fall 2020, their math achievement was greater than only 30% of the pre-pandemic 
sample. 


Math achievement declines were particularly evident for students in 4 and 5 grades in 
fall 2020. 


Overall, math achievement, when benchmarked against a national sample from before the pandemic, 
was lower in fall 2020 than in fall 2019 for nearly all grade levels examined in this report. However, the 
largest drops (ranging from 3 to 13 percentile points) were observed in the earliest grade levels we 
examined, corresponding to the late elementary school period. AIAN, Latinx, and Native Hawaiian 
student groups were most affected, followed by Black and Asian student groups. Put simply, BIPOC 
students appear to have lost significant ground in math during COVID-19 compared to national norms 
for achievement. 


Figure 1. Math percentile rank change by cohort and grade level for BIPOC student groups 
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Note. The blue circle represents the pre-pandemic (fall 2019) median percentile rank for a given group, and the arrows 
represent change in student percentile rank for a given grade span during the pandemic. 


The results for reading are shown in Figure 2. Drops in reading percentile rank during the pandemic are 
relatively small in magnitude overall, especially compared to math. However, we see a similar pattern of 


larger drops for students in 5‘ and 6" grade in fall 2020, as well as larger declines for AIAN, Black, 
Latinx, and Native Hawaiian students. 


Figure 2. Reading percentile rank change by cohort and grade level for BIPOC student groups 
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Note. The blue circle represents the pre-pandemic (fall 2019) median percentile rank for a given group, and the arrows 
represent change in student percentile rank for a given grade span during the pandemic. 


All BIPOC student groups made learning gains during the pandemic, however Black, AIAN, and 
Latinx students only made 55-66% of the typical gains in math and 73-84% of the typical gains in 
reading during the start of the pandemic. 


To address the second research question, we calculated the gains in BIPOC students’ test scores 
(what we call learning gains) between fall of 2019 and fall of 2020 and compared these gains to those 
of a separate pre-pandemic cohort of BIPOC students (assessed in the fall of 2018 and the fall of 
2019). Thus, we can ascertain whether BIPOC students made learning gains since the onset of the 
pandemic and also situate these gains relative to what we would expect for similar students in a more 
typical year. Figure 3 displays the results from this analysis. First, all BIPOC student groups made 
achievement gains from fall 2019 to fall 2020. Most students made between 60-70% of the pre- 
pandemic math gains for their group on average, and Asian American students made 88% of the pre- 
pandemic math gains. In reading, all groups made at least 70% of their historical gains; AIAN and 
Native Hawaiian students made 84-86% and Asian students gained equivalently to students in 2018- 
2019. Comparisons between each group’s learning gains during the pandemic and the national NWEA 
MAP Growth norms for the same period are presented in the appendix. 


Figure 3. Learning gains relative to typical pre-pandemic learning gains 
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historical gains (fall 2018 — fall 2019) calculated for students within the same race/ethnicity, grade, and subject. Weighted 
averages were calculated across grades. AlAN=American Indian and Alaska Native. 


Learning gains during the pandemic were further disaggregated by student gender (Figure 4) and 
school poverty level (Figure 5). In both subjects, male BIPOC students made a lower percentage of 
pre-pandemic gains compared with female BIPOC students. Large differences by school poverty, 
particularly in reading, were also observed. BIPOC students in low poverty schools made over 90% of 
pre-pandemic reading gains between fall 2019 and fall 2020, whereas BIPOC students attending high 
poverty schools made only 65-81% of their historical reading gains. 


Figure 4. Learning gains relative to typical pre-pandemic learning gains by gender 
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Note. Percentage of historical gains was calculated by dividing the average pandemic learning gains (fall 2019 — fall 2020) by 
historical gains (fall 2018 — fall 2019) calculated for students within the same race/ethnicity, gender, grade, and subject. 
Weighted averages were calculated across grades. 


Figure 5. Learning gains relative to typical pre-pandemic learning gains by school poverty level 
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Note. Percentage of historical gains was calculated by dividing the average pandemic learning gains (fall 2019 — fall 2020) by 
historical gains (fall 2018 — fall 2019) calculated for students within the same race/ethnicity, school poverty level, grade, and 
subject. Low poverty schools had <25% of students eligible for free or reduced priced lunch (FRPL) in 2018-19, whereas high 
poverty schools served >75% FRPL-eligible students. Weighted averages were calculated across grades. 


Summary of findings 


The COVID-19 pandemic has exacerbated longstanding educational inequities for BIPOC communities 
in the United States. BIPOC students were less likely to be learning in person and more likely to run 
into obstacles in accessing instruction than White students during the 2020-21 school year. Further, 
differences in health outcomes, food insecurity, and mental health due to systemic racism faced by 
BIPOC students may well have contributed to diminished academic gains during the pandemic. Our 
initial achievement results confirm these challenges. For example, Black, AIAN, and Latinx students 
only made 60-70% of their typical gains in math and 75-90% of typical gains in reading. Our data 
cannot speak to why academic gains were lower during the pandemic, but it is likely that the myriad 
barriers that BIPOC students have faced during the pandemic have impeded their opportunities to 
learn, and as a result they are not progressing at the same rate they were before the pandemic. 
Therefore, a holistic approach to recovery is warranted and below we offer policy recommendations 
meant to encompass a wide range of ways to better support BIPOC students in this moment. 


Understanding the initial educational impacts of COVID-19 on communities of color 


A “return to normal”? Setting recovery goals for BIPOC students 


We use pre-pandemic patterns of achievement and growth to contextualize BIPOC students’ achievement 
and growth during the pandemic. We do not, however, use a typical school year as an aspirational goal for 
students exposed to the pandemic as pre-pandemic educational opportunities were highly inequitable for 
BIPOC students. School systems should use this period to fundamentally rethink how programs, policies, 
and opportunities are allocated to BIPOC students. 


Policy recommendations for supporting COVID-19 recovery for 
BIPOC students 


e Improve remote learning for BIPOC students: 

o Ensure that every student has access to broadband, devices, and the technical support 
necessary to access remote learning opportunities; this continues to be important as it is 
possible that some BIPOC students may remain remote in the 2021-2022 school year. 

o Engage parents and families through regular, well-timed communication that also allows 
families to communicate what they are seeing at home. 

o Provide professional development for teachers and school leaders to support students 
and families in engaging with remote instruction effectively. 


e Extend learning time for BIPOC students: 
o Offer engaging summer school or enrichment opportunities during the summer. 
o Provide additional instructional time by starting the 2021-22 school year early or 
extending the school day. 
o Launch high-quality tutoring programs for BIPOC students tied to school curriculum and 
instruction. 


e Target federal resources to BIPOC students 
o Distribute resources equitably across schools within and across districts; districts with 
higher concentrations of BIPOC students and low-income students should receive a 
higher per-pupil allocation than Whiter and wealthier districts; same is true for schools 
within a district. 
o Engage community-based organizations with a proven track record of serving BIPOC 
communities to help with identifying needs and deploying resources. 


e Address BIPOC students’ academic, social, and emotional needs 

o Evaluate academic achievement and growth regularly using valid, reliable, and 
comparable assessments to identify unfinished learning and learning acceleration. 

o Ensure we use multiple data points to help tell the whole picture. In some places, data 
has been used to underinvest when it should be used to target more resources into 
certain places. We have to use this information to restructure our investments so the 
communities that need more resources get them. 

o Celebrate BIPOC students’ and communities’ strengths and successes, especially those 
that do not show up in traditional measures of student or school performance. 

o Replicate best practices and successful interventions across schools and districts where 
BIPOC students are experiencing greater-than-average gains. 


o Deploy tools (like surveys) that measure students’ social and emotional well-being. 

o Screen regularly for physical and mental health conditions; provide school medical staff 
(nurses) and counselors with training to identify and treat trauma, stress, and anxiety 
effectively. 

o Use federal funds to provide additional mental health and counseling resources to BIPOC 
students and families that have been disproportionately affected by COVID-19. 

o Continue to expand school meal programs by reducing barriers for qualification, 
extending the hours and locations, and providing meals during non-traditional periods 
(e.g. Summer). 
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Appendix 
Terminology & considerations 


NWEA student-level racial and ethnic identifiers are collected annually through class rosters. Such 
identifiers include “Hispanic,” which generally refers to persons who speak Spanish natively and/or 
have Spanish-speaking ancestry. For this report, we adopted the contemporary gender-neutral term 
Latinx for individuals whose origin or ancestry is in Latin America. 


We argue that many structural inequalities (e.g., access to technology infrastructure and/or digital 
devices) disproportionately impacted BIPOC communities, and we also chose to compare how students 
are performing academically relative to same race/ethnic students before and after the pandemic for 
this report’s analyses. In line with Critical Race and Critical Whiteness Theorists, we did not want to 
position whiteness as the default or as centric in our comparisons. Therefore, we have chosen not to 
include White students as a comparison point in these analyses presented in this report. 


Finally, we want to acknowledge how this study is among a large body of research pointing to 
educational inequalities, sometimes referred to as the racial achievement gap. Instead, we hope it is 
used to advance Gloria Ladson-Billings’ nearly 15-year-old challenge for “education research to 
reconceptualize this notion of the achievement gap and to begin to think about the incredible debt that 
we as a nation have accumulated. So rather than focusing on telling people to catch up, we have to 
think about how we, all of us, will begin to pay down this mountain of debt that we have amassed at the 
expense of entire groups of people and their subsequent generations.” Ladson-Billings’ collective call to 
action is more pressing today than ever. 


Sample 


The data for the proposed study are from the NWEA anonymized longitudinal student achievement 
database. School districts use the NWEA MAP Growth assessments to monitor elementary and 
secondary students’ reading and math growth, with assessments typically administered in the fall 
(usually between August and November), winter (usually December to March), and spring (late March 
through June). In total, our sample consists of approximately 2.7 million 3'¢-8'" grade students in 17,000 
public schools who took MAP Growth reading and math assessments across the 2018-19, 2019-20, 
and 2020-21 school years. K-2 students were excluded from this sample as prior research has raised 
concerns that remotely administered tests are not comparable to in-person test in the early grades. Our 
analyses in this study are restricted to BIPOC students from the following groups: AIAN, Asian 
American, Black, Latinx, and Native Hawaiian. 


In this study, we follow longitudinal cohorts of students across the two school years impacted by the 
pandemic (fall 2019-fall 2020) and compare them to students who went through the same grades in the 
prior school years (fall 2018-fall 2019). This design allows for us to understand growth during the 
pandemic in reference to a “typical” (pre-pandemic) period for students of the same racial/ethnic group. 
Table 1 illustrates the cohort design used in this study. 


In a prior analysis of these data, we found that BIPOC students and historically lower-achieving 
students who had tested in previous years were less likely to have taken the MAP Growth assessments 
in fall 2020. Therefore, focusing only on students who had complete data before and after the pandemic 
may underestimate the impact of the COVID-19 disruptions on learning. In the analyses presented in 
this report, we retained students that tested at any point during the three school years (2018-19, 2019- 
20, and 2020-21), even if they did not test during the pandemic period. To address the issue of missing 
test scores, we used students’ past test scores, student characteristics (racial/ethnic group identification 


and biological sex), and school characteristics (percent of students below poverty line, percent of 
students receiving free-and-reduced price lunch, percent White, percent Black, total enrollment, total 
number of students with an Individualized Education Program, geographical location — city, rural, or 
suburban, and percent of schools closed in the larger district) to impute missing scores and to control 
for differences between students tested in fall 2020 and those tested in prior years. We used the mice 
package in R, which imputes incomplete multivariate data using chained equations. The mice package 
appropriately handles and imputes values depending on the variable type, employing a fully conditional 
specification. For each data file, we imputed student MAP Growth scores separately by grade cohort 
and by region of the U.S. to account for regional similarities in schooling and responses to COVID-19. 
Five sets of plausible (imputed) values were created for each data file (for example, five datasets 
corresponding to math scores from 7" graders in Fall 2020). 


Counts of the number of students testing per time point within each cohort and racial/ethnic group, as 
well as the final sample size per group, are presented in Table 2. Overall, the sample is 4% AIAN, 10% 
Asian American, 41% Black, 44% Latinx, and 1% Native Hawaiian. Table 3 presents descriptive 
information on the sample of schools. Public schools that administered MAP Growth were included in 
the sample if least five BIPOC students tested in grade 3-8 during one of the three school years (2018- 
19, 2019-20, and 2020-21). 


Measure of achievement 


Student test scores from the NWEA MAP Growth reading and math assessments were used in this 
study. MAP Growth is a computer adaptive test that precisely measures achievement even for students 
above or below grade level and is vertically scaled to allow for the estimation of gains across time. The 
MAP Growth assessments are typically administered three times a year (fall, winter, and spring) and 
are aligned to state content standards. Test scores are reported on the RIT (Rasch unlT) scale, which 
is a linear transformation of the logit scale units from the Rasch item response theory model. 


In this study, we used both students’ RIT scores and their achievement percentile scores. Achievement 
percentile scores were calculated using the NWEA 2020 MAP Growth norms, which were calculated 
based on a pre-pandemic sample of students from the 2015-16, 2016-17, and 2017-18 school years. 
This national norming sample was used as the reference point for all timepoints examined in this study 
(e.g., fall 2018, fall 2019, and fall 2020). 


Students’ percentile rank is calculated conditional on each student’s grade, subject, and instructional 
week of testing (i.e., the week in the school calendar in which a student tested). Instructional weeks 
were calculated for each student based on their school start date and the individual student’s testing 
dates (for more details on the calculation of instructional weeks, see the norms study). On average, 
students’ fall test occurred within the 3° -4" instructional week of the school year. There were not 
substantial differences observed in average instructional week across school years. 


Within each grade and subject, let Y;, be a student /’s RIT score at instructional week ¢. The predicted 
mean (Y,) and standard deviation (SD(¥,)) for a given grade/subject/instructional week combination 
were pre-calculated based on the NWEA norms model (see Chapter 4 of the norms report). Based on 
these values, we calculated a standardized estimate of the student’s RIT score: 


(Yit- Yt) 


2(Yit) = SD(Y) * 


From the standardized score, we calculated the score percentile (e.g., the proportion of the distribution 
that the student scored as well as, or higher than): 


Mt 
p,(Vie) = PrWe Sy) = | (z)dz, 


where o(z) represents the probability density function. The student normative percentile used in this 
study was scaled to range from 1 to 99: 


Perc = 100 X p, (Viz). 


A 4" grade student who achieved a percentile rank of 10 in math in fall 2019 scored at or above 10% of 
the 4" grade students in the norming sample who had tested in the same instructional week. 


Methods 


RQ1: How does the academic achievement of BIPOC students during the pandemic compare to 
national pre-pandemic patterns of achievement? 


To address the first research question, we calculated the mean and standard deviation of students’ RIT 
scores and median percentile rank across two falls (fall 2018 — fall 2019 for the pre-pandemic cohort, 
fall 2019 — fall 2020 for the pandemic cohort) each subject and student race/ethnicity group. The RIT 
estimates are reported in Tables 4 and 5. The changes in median percentile rank between fall 2019 and 
fall 2020 are shown in Figures 1 and 2. 


RQ2: How do learning gains of BIPOC students from fall 2019 to fall 2020 compare to BIPOC 
student gains in prior years? 


To address this research question, we calculated the average fall-fall gain for each cohort, subject, and 
student race/ethnicity group. For students in the first (pre-pandemic) cohort, these gains were 
estimated from fall 2018-fall 2019. For the pandemic cohort, gains were calculated from fall 2019-fall 
2020. The grade-specific math plots are presented in Figure A1 and reading results are in Figure A2. In 
all grades and race/ethnicity groups, students made smaller gains in math during the COVID-19 
pandemic than students had made in the prior cohort. Overall reading gains for the fall 2019-fall 2020 
cohort were only slightly reduced as compared to gains for the prior cohort of students. 


In addition, to quantify the percentage of historical gains made by students during the COVID-19 
pandemic, we divided the fall-fall gains during the pandemic by the gains of their same-grade same- 
race/ethnicity peers from the prior year. For instance, if the average gain made by the pandemic cohort 
was 9 RIT points but the pre-pandemic cohort gained 10 RIT points on average, the pandemic cohort 
made 90% of the typical gains exhibited by the pre-pandemic cohort. We took a weighted average 
across grades and reported an overall percentage of historical gains by subject and race/ethnicity in 
Figure 3. 


Additionally, these results were further disaggregated by gender (male vs. female students) and school 
poverty level (low poverty determined as <25% of students were eligible for free or reduced priced 
lunch [FRPL] in 2018-19, high poverty as >75% of students were FRPL-eligible). These results are 


presented in Figures 4 and 5. Substantial differences were observed across groups favoring female 
students and students attending low poverty schools. 


Finally, we also calculated the percentage of historical gains made by students during the COVID-19 
pandemic using NWEA national norms for fall-fall gains as the baseline (e.g., Table B.1 and B.3 from 
the 2020 NWEA MAP Growth norms). These results are presented in Figure A3. Overall, similar 
patterns were observed, though Asian students’ gains were observed to be over 100% of the national 
expected growth for students in a given grade span. 


Limitations 


This study is not without limitations. First, we have a combination of in-person and remote testers in our 
sample. While we have preliminary evidence that MAP Growth grade 3-8 tests administered remotely 
are comparable to in-person administration, it is possible that we are not getting a full picture of student 
performance that we would have if all students tested in-person this year. Second, the NWEA sample 
draws from approximately 18,000 schools across the country but is not nationally representative. Third, 
we utilize multiple imputation to recover missing test score data for students that did not test during one 
or more of the school terms. This approach is valid in the case that these scores are missing at 
random, which means that the reason that a test score is missing is not due to the score the student 
would have received had they took the test. Instead, the score is missing at random after we adjust for 
the observed covariates we included in the imputation procedure. Fourth, our sample size for Native 
Hawaiian students is relatively small, which limits the generalizability of the findings for this group. 


Table 1. Cohort design used in the study 


Grade ___ Fall 2018 Fall 2019 Fall 2020 


f 


8 
Note. C1=pre-pandemic cohort followed in a grade pair from Fall 2018-Fall 2019, C2=pandemic cohort followed from Fall 
2019-Fall 2020). Note that the C2 cohort of students followed across grades 7-8 during the pandemic are the same students 
who are in the C1 pre-pandemic cohort in grades 6-7 between Fall 2018-Fall 2019. 


Table 2. Overall sample size per race/ethnicity and cohort as well as sample size per term 


Prior Sample size per term 
Grade Grade Grade Cohort Pandemic 
(18- (19- (20- (fall 18- Cohort (fall 

Race/ethnicity 19) 20) 21) fall 19) —19-fall 20) Total N F18 Ww19 F19 W20 F20 Ww21 
Asian American 2 3 4 — 3-4 43,107 30,435 29,370 29,981 28618 18,790 13,949 
Black 2 3 4 — 3-4 170,463 124,734 123,270 118,473 116,070 69,174 55,715 
Native Hawaiian 2 3 4 — 3-4 2,786 1,804 1,720 1,980 1,961 1,285 1,093 
Latinx 2 3 4 — 3-4 177,670 123,465 126,086 124,910 127,124 83,479 72,275 
AIAN 2 3 4 — 3-4 15,087 10,560 10,093 10,703 10,378 5,057 3,988 
Asian American 3 4 5 3-4 4-5 44,617 34,038 32,839 30,113 28559 19,895 14,789 
Black 3 4 5 3-4 4-5 173,279 129,295 129,404 117,410 114,247 70,431 56,746 
Native Hawaiian 3 4 5 3-4 4-5 2,779 2,063 2,027 2,032 2,019 1,299 1,144 
Latinx 3 4 5 3-4 4-5 185,854 136,548 137,519 126,666 128,596 88,296 76,148 
AIAN 3 4 5 3-4 4-5 14,951 11,466 11,005 10520 10,143 4,975 4,014 
Asian American 4 5 6 4-5 5-6 46,710 33,392 32,205 30,973 28,810 18,187 13,152 
Black 4 5 6 4-5 5-6 180,977 129,861 129,132 122,321 119,656 60,500 49,539 
Native Hawaiian 4 5 6 4-5 5-6 2,765 1,892 1,836 2,052 1,974 915 762 
Latinx 4 5 6 4-5 5-6 196,040 132,962 134,417 134,068 136,126 72,299 61,563 
AIAN 4 5 6 5-6 5-6 15,770 11,819 11,351 11,033 10,489 4,859 3,520 
Asian American 5 6 7 5-6 6-7 47,278 33,906 32,387 31,204 25838 17,611 11,468 
Black 5 6 7 5-6 6-7 181,308 131,849 131,435 116,627 107,793 58,471 46,519 
Native Hawaiian 5 6 7 5-6 6-7 2,628 1,866 1,851 1,681 1,527 894 773 
Latinx 5 6 7 5-6 6-7 195,076 139,570 140,816 124,863 118,673 71,319 58,401 
AIAN 5 6 7 5-6 6-7 16,493 12,133 11,651 11,076 9,988 4,689 3,426 
Asian American 6 7 8 6-7 7-8 42,811 33,369 29,804 29,086 24502 17,096 11,381 
Black 6 7 8 6-7 7-8 168,039 125,123 120,764 113,049 104,309 58,489 47,319 
Native Hawaiian 6 7 8 6-7 7-8 2,475 1,829 1,651 1,715 1,561 902 770 
Latinx 6 7 8 6-7 7-8 186,080 139,398 131,567 125,341 118,030 71,909 59,262 
AIAN 6 7 8 6-7 7-8 15,583 11,513 10,458 10,321 8,766 4,634 3,237 


Note. Math sample sizes are reported in this table but the final sample sizes for reading were very similar. 


Table 3. Characteristics of Our School Samples (By Subject) Relative to US Public Schools 


Serving Grades 3-8 


Total School Enrollment 


Mathematics 


486.5 (259.20) 


Readin 


486.1 (258.50) 


US Schools 


486.84 (326.57) 


Percent Black 0.15 (0.24) 0.15 (0.24) 0.15 (0.24) 
Percent Latinx/Hispanic 0.21 (0.25) 0.21 (0.24) 0.26 (0.29) 
Percent Asian 0.04 (0.08) 0.04 (0.08) 0.04 (0.09) 
Percent White 0.58 (0.33) 0.58 (0.33) 0.53 (0.34) 
Percent FRPL 0.54 (0.29) 0.54 (0.28) 0.56 (0.28) 
Percent ELL 0.08 (0.09) 0.08 (0.09) 0.09 (0.11) 
Percent SPED 0.14 (0.03) 0.14 (0.03) 0.13 (0.04) 
School Location* 
City 0.29 (0.45) 0.28 (0.45) 0.28 (0.45) 
Rural 0.27 (0.44) 0.27 (0.44) 0.28 (0.45) 
Suburb 0.33 (0.47) 0.33 (0.47) 0.33 (0.47) 
Town 0.12 (0.32) 0.12 (0.32) 0.12 (0.32) 
ros onder aaed 0.39 (0.30) 0.39 (0.30) 0.30(.33) 
N= 18,079 N= 18,152 N= 73,345 


Note. FRPL=free or reduced-price lunch, ELL=English Language Learners, SPED=special education students. “Because of 
rounding, may add to greater than 100%. The racial composition, FRPL, and school urbanicity variables came from the NCES 
Common Core of Data (CCD), the percent ELL and SPED variables come from the Civil Rights Data Collection, and the 
school closure variable refers to school closures in Sept. 2020 and comes from the U.S. School Closure & Distance Learning 
Database (see documentation for more detail on how school closures are quantified). 


Table 4. Means and standard deviations for math achievement by cohort and race/ethnicity 


Grade First Fall Term Second Fall Term Difference 
Race/ethnicity span Cohort N M SD M SD M SE 
AIAN 3rd-4th 2018-2019 4,402 182.7 13.6 194.2 13.8 11.5 0.3 
AIAN 3rd-4th 2019-2020 3,974 183.1 12.5 192.3 13.0 9.2 0.3 
AIAN 4th-5th 2018-2019 4,062 193.9 14.5 202.8 15.2 8.9 0.3 
AIAN 4th-5th 2019-2020 4,402 194.2 13.8 200.8 15.0 6.7 0.3 
AIAN 5th-6th 2018-2019 3,869 203.4 15.0 208.7 14.2 5.3 0.3 
AIAN 5th-6th 2019-2020 4,052 202.8 15.2 206.1 14.2 3.3 0.3 
AIAN 6th-7th 2018-2019 8,956 207.7 14.9 213.0 15.0 5.3 0.2 
AIAN 6th-7th 2019-2020 9,161 207.2 13.8 212.7 23.5 5.5 0.3 
AIAN 7th-8th 2018-2019 8,433 213.6 16.1 217.8 16.0 4.2 0.2 
AIAN 7th-8th 2019-2020 8,956 213.0 15.0 217.8 25.1 4.9 0.3 
Asian American 3rd-4th 2018-2019 15,692 196.8 13.7 209.6 14.0 12.9 0.2 
Asian American 3rd-4th 2019-2020 14,209 196.7 13.0 208.9 14.2 12.1 0.2 
Asian American 4th-5th 2018-2019 13,362 209.9 14.7 220.9 15.4 11.1 0.2 
Asian American 4th-5th 2019-2020 15,685 209.6 14.0 219.9 15.4 10.3 0.2 
Asian American 5th-6th 2018-2019 12,106 220.3 15.9 225.6 14.3 5.3 0.2 
Asian American 5th-6th 2019-2020 13,350 220.9 15.4 225.5 14.5 4.6 0.2 
Asian American 6th-7th 2018-2019 25,652 225.5 16.7 233.9 17.6 8.4 0.2 
Asian American 6th-7th 2019-2020 24,394 226.0 15.0 233.0 22.2 7.0 0.2 
Asian American 7th-8th 2018-2019 22,093 233.2 18.7 239.9 19.0 6.7 0.2 
Asian American 7th-8th 2019-2020 25,652 233.9 17.6 238.8 27.5 4.9 0.2 
Black 3rd-4th 2018-2019 58,768 183.5 12.6 194.7 12.9 11.2 0.1 
Black 3rd-4th 2019-2020 56,644 183.5 12.0 192.5 12.9 9.0 0.1 
Black 4th-5th 2018-2019 50,899 194.8 13.3 203.4 13.7 8.6 0.1 
Black 4th-5th 2019-2020 58,679 194.8 12.9 201.1 13.5 6.3 0.1 
Black 5th-6th 2018-2019 49,588 203.3 14.5 207.9 12.9 4.5 0.1 
Black 5th-6th 2019-2020 50,812 203.4 13.7 206.3 12.9 2.9 0.1 
Black 6th-7th 2018-2019 96,091 207.2 14.6 212.6 15.0 5.5 0.1 
Black 6th-7th 2019-2020 91,988 207.1 12.8 212.2 19.1 5.1 0.1 
Black 7th-8th 2018-2019 85,994 212.9 16.0 217.6 15.6 4.7 0.1 
Black 7th-8th 2019-2020 96,091 212.6 15.0 217.1 21.8 4.4 0.1 
Latinx 3rd-4th 2018-2019 75,843 185.1 12.7 196.9 13.2 11.8 0.1 
Latinx 3rd-4th 2019-2020 71,868 184.9 12.2 193.9 12.7 9.0 0.1 
Latinx 4th-5th 2018-2019 60,046 197.2 13.3 206.2 14.0 8.9 0.1 
Latinx 4th-5th 2019-2020 75,810 196.9 13.2 203.2 13.6 6.3 0.1 
Latinx 5th-6th 2018-2019 60,052 206.3 14.5 210.7 13.0 4.4 0.1 
Latinx 5th-6th 2019-2020 60,019 206.2 14.0 208.3 12.9 2.2 0.1 
Latinx 6th-7th 2018-2019 115,484 210.7 14.1 216.1 15.5 5.4 0.1 
Latinx 6th-7th 2019-2020 107,541 210.2 13.2 214.7 19.3 4.4 0.1 


Latinx 7th-8th 2018-2019 103,708 216.4 16.0 220.9 16.5 44 0.1 


Latinx 7th-8th 2019-2020 115,484 216.1 15.5 219.7 22.1 3.6 0.1 
Native Hawaiian = 3rd-4th 2018-2019 1,016 185.8 13.7 196.8 13.6 11.1 0.6 
Native Hawaiian = 3rd-4th 2019-2020 984 185.3 12.2 193.9 12.5 8.6 0.6 
Native Hawaiian = 4th-5th 2018-2019 680 196.4 14.6 205.7 14.9 9.3 0.8 
Native Hawaiian  4th-5th 2019-2020 1,015 196.8 13.5 203.1 13.9 6.2 0.6 
Native Hawaiian = 5th-6th 2018-2019 667 205.9 15.9 210.2 14.8 4.3 0.8 
Native Hawaiian = 5th-6th 2019-2020 679 205.7 14.9 207.6 13.4 1.9 0.8 
Native Hawaiian —‘6th-7th 2018-2019 1,478 209.4 16.0 215.2 16.5 5.7 0.6 
Native Hawaiian —6th-7th 2019-2020 1,387 209.8 14.4 215.0 21.7 5.2 0.7 
Native Hawaiian = 7th-8th 2018-2019 1,301 216.1 17.8 220.3 17.4 4.2 0.7 
Native Hawaiian _7th-8th 2019-2020 1,478 215.2 16.5 219.6 25.4 4.4 0.8 


second fall is fall 2019. For the 2019-2020 (pandemic) cohort, the first fall is fall 2019 and the second fall is fall 2020. 
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Table 5. Means and standard deviations for reading achievement by cohort and race/ethnicity 


Grade First Fall Term Second Fall Term Difference 
Race/ethnicity span Cohort N M SD M SD M SE 
AIAN 3rd-4th 2018-2019 4,361 180.8 16.3 190.4 16.1 9.7 0.3 
AIAN 3rd-4th 2019-2020 3,897 181.1 15.0 191.5 15.4 10.4 0.3 
AIAN 4th-5th 2018-2019 3,978 190.5 16.7 197.7 16.5 7.2 0.4 
AIAN 4th-5th 2019-2020 4,359 190.5 16.1 198.2 16.4 7.7 0.3 
AIAN 5th-6th 2018-2019 3,768 198.7 16.6 204.6 15.3 5.9 0.4 
AIAN 5th-6th 2019-2020 3,972 197.7 16.5 204.9 15.5 7.2 0.4 
AIAN 6th-7th 2018-2019 8,845 203.1 16.1 207.0 15.2 3.9 0.2 
AIAN 6th-7th 2019-2020 9,173 203.0 14.9 209.4 24.2 6.4 0.3 
AIAN 7th-8th 2018-2019 8,408 207.4 16.0 210.8 15.3 3.4 0.2 
AIAN 7th-8th 2019-2020 8,845 207.0 15.2 213.1 23.9 6.1 0.3 
Asian American 3rd-4th 2018-2019 15,680 195.3 15.7 205.0 14.6 9.6 0.2 
Asian American 3rd-4th 2019-2020 14,051 194.6 14.6 205.4 13.6 10.8 0.2 
Asian American 4th-5th 2018-2019 13,224 205.3 15.3 212.5 14.5 7.2 0.2 
Asian American 4th-5th 2019-2020 15,671 205.0 14.6 212.8 13.7 7.9 0.2 
Asian American 5th-6th 2018-2019 12,885 212.9 15.1 219.2 13.1 6.3 0.2 
Asian American 5th-6th 2019-2020 13,216 212.5 14.5 219.1 12.8 6.6 0.2 
Asian American 6th-7th 2018-2019 25,821 217.9 15.3 222.1 14.6 4.2 0.1 
Asian American 6th-7th 2019-2020 24,175 217.9 13.8 223.9 18.0 6.0 0.1 
Asian American 7th-8th 2018-2019 23,553 222.3 15.5 226.0 14.9 3.7 0.1 
Asian American 7th-8th 2019-2020 25,821 222.1 14.6 228.0 19.3 5.8 0.2 
Black 3rd-4th 2018-2019 58,529 182.9 15.6 192.6 15.0 9.7 0.1 
Black 3rd-4th 2019-2020 56,262 182.7 14.6 192.2 14.9 9.5 0.1 
Black 4th-5th 2018-2019 49,786 192.9 15.6 199.9 14.7 7.0 0.1 
Black 4th-5th 2019-2020 58,422 192.6 15.0 199.0 14.8 6.3 0.1 
Black 5th-6th 2018-2019 48,277 199.9 15.6 205.5 14.1 5.5 0.1 
Black 5th-6th 2019-2020 49,703 199.9 14.7 205.0 14.5 5.1 0.1 
Black 6th-7th 2018-2019 96,800 204.7 15.7 208.4 14.5 3.7 0.1 
Black 6th-7th 2019-2020 95,252 204.7 14.1 209.0 20.6 4.3 0.1 
Black 7th-8th 2018-2019 87,925 209.0 15.6 212.5 14.4 3.4 0.1 
Black 7th-8th 2019-2020 96,800 208.4 14.5 212.7 21.2 4.2 0.1 
Latinx 3rd-4th 2018-2019 71,198 183.0 15.8 193.3 15.5 10.2 0.1 
Latinx 3rd-4th 2019-2020 65,322 182.7 14.8 192.0 14.9 9.3 0.1 
Latinx 4th-5th 2018-2019 56,306 193.2 16.0 200.5 15.2 7.3 0.1 
Latinx 4th-5th 2019-2020 71,161 193.3 15.5 199.3 15.0 6.1 0.1 
Latinx 5th-6th 2018-2019 56,435 200.8 15.8 206.4 14.7 5.6 0.1 
Latinx 5th-6th 2019-2020 56,282 200.5 15.2 205.3 14.4 4.8 0.1 


Latinx 6th-7th 2018-2019 110,142 206.2 15.7 209.6 15.5 3.4 0.1 


Latinx 6th-7th 2019-2020 102,459 205.8 14.8 209.5 20.1 3.7 0.1 
Latinx 7th-8th 2018-2019 99,934 210.3 15.9 213.3 15.6 3.0 0.1 
Latinx 7th-8th = 2019-2020 110,142 209.6 15.5 213.5 21.0 3.9 0.1 
Native Hawaiian 3rd-4th 2018-2019 1,009 184.6 16.0 193.9 15.2 9.3 0.7 
Native Hawaiian 3rd-4th 2019-2020 979 183.6 15.4 193.0 15.0 9.3 0.7 
Native Hawaiian Ath-5th 2018-2019 683 191.9 17.5 200.1 15.7 8.2 0.9 
Native Hawaiian Ath-5th 2019-2020 1,009 193.9 15.2 200.6 14.5 6.7 0.7 
Native Hawaiian Sth-6th 2018-2019 668 200.3 17.3 206.2 15.2 5.9 0.9 
Native Hawaiian Sth-6th 2019-2020 682 200.0 15.7 205.2 14.9 5.2 0.8 
Native Hawaiian 6th-7th 2018-2019 1,471 205.6 17.2 209.8 15.8 4.2 0.6 
Native Hawaiian 6th-7th 2019-2020 1,366 205.6 15.1 211.2 20.5 5.6 0.7 
Native Hawaiian 7th-8th 2018-2019 1,385 209.4 16.9 213.0 15.4 3.7 0.6 
Native Hawaiian 7th-8th 2019-2020 1,471 209.8 15.8 214.0 22.8 4.3 0.7 


second fall is fall 2019. For the 2019-2020 (pandemic) cohort, the first fall is fall 2019 and the second fall is fall 2020. 
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Figure A1. Math test score gains by cohort and grade level for BIPOC student groups 
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Figure A2. Reading test score gains by cohort and grade level for BIPOC student groups 
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Figure A3. Fall-fall gains during the COVID-19 pandemic relative to NWEA national norms for 
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Note. Percentage of national norms gains was calculated by dividing the average pandemic (Fall 2019 — Fall 2020) learning 
gains for a given group by the national average for fall-fall gains based on a national pre-pandemic sample of students in a 
grade span and subject (e.g., Table B.1 and B.3 from the 2020 NWEA MAP Growth norms). Weighted averages were 
calculated across grades. AlAN=American Indian and Alaska Native. 
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nwea 


NWEA is a not-for-profit organization that supports students and educators worldwide by providing assessment solutions, insightful 
reports, professional learning offerings, and research services. Visit NWEA.org to find out how NWEA can partner with you to help all 


kids learn. 


2021 NWEA. NWEA and MAP are registered trademarks, and MAP Growth is a trademark, of NWEA in the US and in other 
countries. All rights reserved. No part of this document may be modified or further distributed without written permission from NWEA. 
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